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[57] ABSTRACT 

A method and system for adaptively filtering a speech signal. 
The method includes decomposing the signal into subbands, 
which may include performing a discrete Fourier transform 
on the signal to provide approximately orthogonal compo- 
nents. The method also includes determining a speech 
quality indicator for each subband, which may include 
estimating a signal-to-noise ratio for each subband. The 
method also includes selecting a filter for filtering each 
subband depending on the speech quality indicator, which 
may include estimating parameters for the filter based on a 
clean speech signal. The method further includes determin- 
ing an overall average error for the filtered subbands, which 
may include calculating a mean-squared error. The method 
still further includes identifying at least one filtered subband 
which, if excluded from the filtered speech signal, would 
reduce the overall average error determined, and combining, 
with exception of the filtered subbands identified, the filtered 
subbands to provide an estimated filtered speech signal. The 
system includes filters and software for performing the 
method. 

16 Claims, 3 Drawing Sheets 
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METHOD AND SYSTEM FOR ADAPTIVE 
FILTERING OF SPEECH SIGNALS USING 
SIGNAL-TO-NOISE RATIO TO CHOOSE 
SUBBAND FILTER BANK 

RELATED APPLICATION 

This application is related to U.S. patent application Ser. 
No. 08/694,654, which was filed on the same date and 
assigned to the same assignee as the present application; Ser. 
No. 08/496,068, which was filed on Jun. 28, 1995; and 
08/722,547, which was filed on Sep. 27, 1996. 

TECHNICAL FIELD 

This invention relates to an adaptive method and system 
for filtering speech signals. 

BACKGROUND ART 

In wireless communications, background noise and static 
can be annoying in speaker to speaker conversation and a 
hindrance in speaker to machine recognition. As a result, 
noise suppression is an important part of the enhancement of 
speech signals recorded over wireless channels in mobile 
environments. 

In that regard, a variety of noise suppression techniques 
have been developed. Such techniques typically operate on 
single microphone, output-based speech samples which 
originate in a variety of noisy environments, where it is 
assumed that the noise component of the signal is additive 
with unknown coloration and variance. 

One such technique is Least Mean-Squared (LMS) Pre- 
dictive Noise Cancelling. In this technique it is assumed that 
the additive noise is not predictable, whereas the speech 
component is predictable. LMS weights are adapted to the 
time series of the signal to produce a time- varying matched 
filter for the predictable speech component such that the 
mean -squared error (MSE) is minimized. The estimated 
clean speech signal is then the filtered version of the time 
series. 

However, the structure of speech in the time domain is 
neither coherent nor stationary enough for this technique to 
be effective. A trade-off is therefore required between fast 
settling time/good tracking ability and the ability to track 
everything (including noise). This technique also has diffi- 
culty with relatively unstructured non-voiced segments of 
speech. 

Another noise suppression technique is Signal Subspace 
(SSP) filtering (which here includes Spectral Subtraction 
(SS)). SSP is essentially a weighted subspace fitting applied 
to speech signals, or a set of bandpass filters whose outputs 
are linearly weighted and combined. SS involves estimating 
the (additive) noise magnitude spectrum, typically done 
during non-speech segments of data, and subtracting this 
spectrum from the noisy speech magnitude spectrum to 
obtain an estimate of the clean speech spectrum. If the 
resulting spectral estimate is negative, it is rectified to a 
small positive value. This estimated magnitude spectrum is 
then combined with the phase information from the noisy 
signal and used to construct an estimate of the clean speech 
signal. 

SSP assumes the speech signal is well-approximated by a 
sum of sinusoids. However, speech signals are rarely simply 
sums of undamped sinusoids and can, in many common 
cases, exhibit stochastic qualities (e.g., unvoiced fricatives). 
SSP relies on the concept of bias-variance trade-off For 
channels having a Signal-to-Noise Ratio (SNR) less than 0 



16,025 

2 

dB, some bias is permitted to give up a larger dosage of 
variance and obtain a lower overall MSE. In the speech case, 
the channel bias is the clean speech component, and the 
channel variance is the noise component. However, SSP 
5 does not deal well with channels having SNR greater than 
zero. 

In addition, SS is undesirable unless the SNR of the 
associated channel is less than 0 dB (i.e., unless the noise 
component is larger than the signal component). For this 

10 reason, the ability of SS to improve speech quality is 
restricted to speech masked by narrowband noise. SS is best 
viewed as an adaptive notch filter which is not well appli- 
cable to wideband noise. 

Still another noise suppression technique is Wiener 

15 filtering, which can take many forms including a statistics- 
based channel equalizer. In this context, the time domain 
signal is filtered in an attempt to compensate for non- 
uniform frequency response in the voice channel. Typically, 
this filter is designed using a set of noisy speech signals and 

20 the corresponding clean signals. Taps are adjusted to opti- 
mally predict the clean sequence from the noisy one accord- 
ing to some error measure. Once again, however, the struc- 
ture of speech in the time domain is neither coherent nor 
stationary enough for this technique to be effective. 

25 

Yet another noise suppression technique is Relative Spec- 
tral (RASTA) speech processing. In this technique, multiple 
filters are designed or trained for filtering spectral subbands. 
First, the signal is decomposed into N spectral subbands 
(currently, Discrete Fourier Transform vectors are used to 
define the subband filters). The magnitude spectrum is then 
filtered with N/2+1 linear or non-linear neural-net subband 
filters. 

However, the characteristics of the complex transformed 

35 signal (spectrum) have been elusive. As a result, RASTA 
subband filtering has been performed on the magnitude 
spectrum only, using the noisy phase for reconstruction. 
However, an accurate estimate of phase information gives 
little, if any, noticeable improvement in speech quality. 

40 The dynamic nature of noise sources and the non- 
stationery nature of speech ideally call for adaptive tech- 
niques to improve the quality of speech. Most of the existing 
noise suppression techniques discussed above, however, are 
not adaptive. While some recently proposed techniques are 

45 designed to adapt to the noise level or SNR, none take into 
account the non-stationary nature of speech and try to adapt 
to different sound categories. 

Thus, there exists a need for an adaptive noise suppression 
technique. Ideally, such a technique would employ subband 

50 filterbank chosen according to the SNR of a channel, inde- 
pendent of the SNR estimate of other channels. By special- 
izing sets of filterbanks for various SNR levels, appropriate 
levels for noise variance reduction and signal distortion may 
be adaptively chosen to minimize overall MSE. 

55 DISCLOSURE OF INVENTION 

Accordingly, it is the principle object of the present 
invention to provide an improved method and system for 
filtering speech signals. 

60 According to the present invention, then, a method and 
system are provided for adaptively filtering a speech signal. 
The method comprises decomposing the speech signal into 
a plurality of subbands, and determining a speech quality 
indicator for each subband. The method further comprises 

65 selecting one of a plurality of filters for each subband, 
wherein the filter selected depends on the speech quality 
indicator determined for the subband, filtering each subband 



11/01/2003, EAST Version: 1.4.1 



5,806,025 



according to the filter selected, and combining the filtered 
subbands to provide an estimated filtered speech signal. 

The system of the present invention for adaptively filter- 
ing a speech signal comprises means for decomposing the 
speech signal into a plurality of subbands, means for deter- 
mining a speech quality indicator for each subband, and a 
plurality of filters for filtering the subbands. The system 
further comprises means for selecting one of the plurality of 
filters for each subband, wherein the filter selected depends 
on the speech quality indicator determined for the subband, 
and means for combining the filtered subbands to provide an 
estimated filtered speech signal. 

These and other objects, features and advantages will be 
readily apparent upon consideration of the following 
detailed description in conjunction with the accompanying 
drawings. 

BRIEF DESCRIPTION OF DRAWINGS 

FIGS. la-b are plots of filterbanks trained at Signal-to- 
Noise Ratio values of 0, 10, 20 dB at subbands centered 
around 800 Hz and 2200 Hz, respectively; 

FIGS. 2a-e are flowcharts of the method of the present 
invention; and 

FIG. 3 is a block diagram of the system of the present 
invention. 

BEST MODE FOR CARRYING OUT THE 
INVENTION 

Traditionally, the Wiener filtering techniques discussed 
above have been packaged as a channel equalizer or spec- 
trum shaper for a sequence of random variables. However, 
the subband filters of the RASTA form of Wiener filtering 
can more properly be viewed as Minimum Mean-squared 
Error Estimators (MMSEE) which predict the clean speech 
spectrum for a given channel by filtering the noisy spectrum, 
where the filters are pre-determined by training them with 
respect to MSE on pairs of noisy and clean speech samples. 

In that regard, original versions of RASTA subband filters 
consisted of heuristic Autoregressive Means Averaging 
(ARMA) filters which operated on the compressed magni- 
tude spectrum. The parameters for these filters were 
designed to provide an approximate matched filter for the 
speech component of noisy compressed magnitude spec- 
trums and were obtained using clean speech spectra 
examples as models of typical speech. Later versions used 
Finite Impulse Response (FIR) filterbanks which were 
trained by solving a simple least squares prediction problem, 
where the FIR filters predicted known clean speech spectra 
from noisy realizations of it. 

Assuming that the training samples (clean and noisy) are 
representative of typical speech samples and that speech 
sequences are approximately stationary across the sample, it 
can be seen that a MMSEE is provided for speech magnitude 
spectra from noisy speech samples. In the case of FIR 
filterbanks, this is actually a Linear MMSEE of the com- 
pressed magnitude spectrum. This discussion can, however, 
be extended to include non-linear predictors as well. As a 
result, the term MMSEE will be used, even as reference is 
made to LMMSEE. 

There are, however, two problems with the above assump- 
tions. First, the training samples cannot be representative of 
all noise colorations and SNR levels. Second, speech is not 
a stationary process. Nevertheless, MMSEE may be 
improved by changing those assumptions and creating an 
adaptive subband Wiener filter which minimizes MSE using 



specialized filterbanks according to speaker characteristics, 
speech region and noise levels. 

In that regard, the design of subband FIR filters is subject 
to a MSE criterion. That is, each subband filter is chosen 

s such that it minimizes squared error in predicting the clean 
speech spectra from the noisy speech spectra. This squared 
error contains two components i) signal distortion (bias); 
and ii) noise variance. Hence a bias-variance trade-off is 
again seen for minimizing overall MSE. This trade-off 

10 produces filterbanks which are highly dependent on noise 
variance. For example, if the SNR of a "noisy" sample were 
infinite, the subband filters would all be simply 5^ where 



I 0, o.w. 



On the other hand, when the SNR is low, filterbanks are 
obtained whose energy is smeared away from zero. This 
phenomenon occurs because the clean speech spectra is 

20 relatively coherent compared to the additive noise signals. 
Therefore, the overall squared error in the least squares 
(training) solution is minimized by averaging the noise 
component (i.e., reducing noise variance) and consequently 
allowing some signal distortion. If this were not true, 

25 nothing would be gained (with respect to MSE) by filtering 
the spectral magnitudes of noisy speech. 

Three typical filterbanks which were trained at SNR 
values of 0, 10, 20 dB, respectively, are shown in FIG. 1 to 
illustrate this point. The first set of filters (FIG. la) corre- 

30 spond to the subband centered around 800 Hz, and the 
second (FIG. lb) represent the region around 2200 Hz. The 
filters corresponding to lower SNR's (In FIG. 1, the filter- 
banks for the lower SNR levels have center taps which are 
similarly lower) have a strong averaging (lowpass) capabil- 

35 ity in addition to an overall reduction in gain. 

With particular reference to the filterbanks used at 2200 
Hz (FIG. lb), this region of the spectrum is a low -point in 
the average spectrum of the clean training data, and hence 
the subband around 2200 Hz has a lower channel SNR than 

40 the overall SNR for the noisy versions of the training data. 
So, for example, when training with an overall SNR of 0 dB, 
the subband SNR for the band around 2200 Hz is less than 
0 dB (i.e., there is more noise energy than signal energy). As 
a result, the associated filterbank, which was trained to 

45 minimize MSE, is nearly zero and effectively eliminates the 
channel. 

Significantly, if the channel SNR cannot be brought above 
0 dB by filtering the channel, overall MSE can be improved 
by simply zeroing the channel. This is equivalent to includ- 

50 ing a filter in the set having all zero coefficients. To pre- 
determine the post-filtered SNR, three quantities are needed: 
i) an initial (pre-filtered) SNR estimate; ii) the expected 
noise reduction due to the associated subband filter; and iii) 
the expected (average speech signal distortion introduced by 

55 the filter. For example, if the channel SNR is estimated to be 
-3 dB, the associated subband filter's noise variance reduc- 
tion capability at 5 dB, and the expected distortion at -1 dB, 
a positive post-filtering SNR is obtained and the filtering 
operation should be performed. Conversely, if the pre- 

60 filtering SNR was instead -5 dB, the channel should simply 
be zeroed. 

The above discussion assumes that an estimator of sub- 
band SNR is available This estimator must be used for the 
latter approach of determining the usefulness of a channel's 
65 output as well as for adaptively determining which subband 
filter should be used. In that regard, an SNR estimation 
technique well known in the art which uses the bimodal 
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characteristic of a noisy speech sample's histogram to 
determine the expected values of signal and noise energy 
may be used. However, accurately tracking multiple 
(subband) SNR estimates is difficult since instantaneous 
SNR for speech signals is a dramatically varying quantity. 
Hence, the noise spectrum, which is a relatively stable 
quantity, may instead be tracked. This estimate may then be 
used to predict the localized subband SNR values. The 
bimodal idea of the known SNR estimation technique 
described above may still contribute as a noise spectrum 
estimate. 

Thus, according to the present invention, speech distor- 
tion is allowed in exchange for reduced noise variance. This 
is achieved by throwing out channels whose output SNR 
would be less than 0 dB and by subband filtering the noisy 
magnitude spectrum. Noise averaging gives a significant 
reduction in noise variance, while effecting a lesser amount 
of speech distortion (relative to the reduction in noise 
variance). Subband filterbanks are chosen according to the 
SNR of a channel, independent of the SNR estimate of other 
channels, in order to adapt to a variety of noise colorations 
and variations in speech spectra. By specializing sets of 
filterbanks for various SNR levels, appropriate levels for 
noise variance reduction and signal distortion may be adap- 
tively chosen according to subband SNR estimates to mini- 
mize overall MSE. In such a fashion, the problem concern- 
ing training samples which cannot be representative of all 
noise colorations and SNR levels is solved. 

Referring now to FIGS. 2a-e, flowcharts of the method of 
the present invention are shown. As seen therein, the method 
comprises decomposing (10) the speech signal into a plu- 
rality of subbands, determining (12) a speech quality indi- 
cator for each subband, selecting (14) one of a plurality of 
filters for each subband, wherein the filter selected depends 
on the speech quality indicator determined for the subband, 
and filtering (16) each subband according to the filter 
selected. At this point, the filtered subbands may simply be 
combined (not shown) to provide an estimated filtered 
speech signal. 

However, the method may farther comprise determining 
(18) an overall average error for a filtered speech signal 
comprising the filtered subbands, and identifying (20) at 
least one filtered subband which, if excluded from the 
filtered speech signal, would reduce the overall average error 
determined. In this embodiment, the method still further 
comprises combining (22), with the exception of the at least 
one filtered subband identified, the filtered subbands to 
provide an estimated filtered speech signal. 

While the subband decomposition described above is 
preferably accomplished by Discrete Fourier Transform 
(DFT), it should be noted that any arbitrary transform which 
well-decomposes speech signals into approximately 
orthogonal components may also be employed (11) (e.g., 
Karhunen-Loeve Transform (KLT)), Likewise, speech qual- 
ity estimation is preferably accomplished using the SNR 
estimation technique previously described where the sub- 
band SNR for each subband in the decomposition is esti- 
mated (13). However, other speech quality estimation tech- 
niques may also be used. 

It should also be noted that, with respect to subband filter 
determination, the estimates of speech quality are used to 
assign a filter to each channel, where the filters are chosen 
from a set of pre-trained filters (15). This set of pre-trained 
filters represents a range of speech quality (e.g., SNR), 
where each is trained for a specific level of quality, with each 
subband channel having its own set of such filters to choose 
from. It can thus be seen that multiple filters are trained for 
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each subband, and the appropriate subband filter is adap- 
tively chosen according to the quality indicator. It should be 
noted that these filters are not necessarily linear and can exist 
as "neural networks" which are similarly trained and chosen. 

5 Still further, with respect to bias-variance trade-off, if the 
quality indicator shows that overall average error could be 
reduced by throwing out a subband channel from the clean 
speech estimate, then that channel is discarded. This trade- 
off is performed after choosing subband filters because the 

10 thresholds for the trade-off are a function of the chosen 
interbank. Remaining outputs of the subband filters are used 
to reconstruct a clean estimate of the speech signal. While 
error is preferably measured according to the mean-squared 
technique (19), other error measures may also be used. 

15 Thus, using quality indicators (e.g., SNR), subband filters 
for subband speech processing are adaptively chosen. If the 
quality indicator is below a threshold for a subband channel, 
the channel's contribution to the reconstruction is thrown 
out in a bias-variance trade-off for reducing overall MSE. 

20 Referring next to FIG. 3, a block diagram of the system 
of the present invention is shown. As seen therein, a cor- 
rupted speech signal (30) is transmitted to a decomposer 
(32). As previously discussed with respect to the method of 
the present invention, decomposer (32) decomposes speech 

25 signal (30) into a plurality of subbands. As also previously 
discussed, such decomposing is preferably accomplished by 
a performing a discrete Fourier transform on speech signal 
(30). However, other transform functions which well- 
decompose speech signal (30) into approximately orthogo- 

30 nal components may also be used, such as a KLT. 

Decomposer (32) generates a decomposed speech signal 
(34), which is transmitted to an estimator (36) and a filter 
bank (38). Once again, as previously discussed with respect 
to the method of the present invention, estimator (36) 

35 determines a speech quality indicator for each subband. 
Preferably, such a speech quality indicator is an estimated 
SNR. 

Depending on the speech quality of the subband, estima- 
tor (36) also selects one of a plurality of filters from filter 

40 bank (38) for that subband, wherein each of the plurality of 
filters is associated with one of the plurality of subbands. As 
previously discussed, the plurality of filters from filter bank 
(38) may be pre-trained using clean speech signals (15). 
Moreover, while any type of estimator (36) well known in 

45 the art may be used, estimator (36) preferably comprises a 
bimodal SNR estimation process which is also used on the 
training data to create valid look-up tables. 

Still referring to FIG. 3, after each frame is filtered at filter 
bank (38) according to the filter selected therefor by esti- 

50 mator (36), a filtered decomposed speech signal (40) is 
transmitted to a reconstructor (42), where the filtered sub- 
bands are combined in order to construct an estimated clean 
speech signal (44). As previously discussed, however, recon- 
structor (42) may first determines an overall average error 

55 for a filtered speech signal comprising the filtered subbands. 
While any technique well known in the art may be used, 
such an overall average error is preferably calculated based 
on MSE. 

Thereafter, reconstructor (42) may identify those filtered 
60 subband which, if excluded from the filtered speech signal, 
would reduce the overall average error. Such filtered sub- 
bands are then discarded, and reconstructor (42) combines 
the remaining filtered subbands in order to construct an 
estimated clean speech signal (44). As those of ordinary skill 
65 in the art will recognize, the system of the present invention 
also includes appropriate software for performing the above- 
described functions. 
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It should be noted that the subband filtering approach of combining, with the exception of the at least one filtered 

the present invention is a generalization of the RASTA subband identified, the filtered subbands to provide an 

speech processing approach described above, as well as in estimated filtered speech signal. 

U.S. Pat. No. 5,450,522 and an article by H. Hermansky et 2. The method of claim 1 wherein decomposing the signal 

al. entitled "RASTA Processing of Speech", IEEE Trans, s into subbands comprises performing a transform on the 

Speech and Audio Proc, October, 1994. Moreover, while si 8 nal t0 provide approximately orthogonal components, 

not adaptive, the foundation for the subband filtering con- 3 * ^ method of claim 2 wherein performing a transform 

cept using trained filterbanks is described in an article by H. on the Sl ^ al comprises performing a discrete Fourier trans- 

Hermansky et al. entitled "Speech Enhancement Based on torm ° D the s , 1 S° al * 

rn m i rj • „ rrrr T^ A ccd / „ rt 4. 1 ne method 01 claim 1 whe rein determining a speech 

Temporal Processing , IEEE ICASSP Conference 10 . . c t. ut. j * * L ■ 1 • 

„ r ~ . ' . nnr „ , c J , quauty indicator for each subband of the signal comprises 

Proceedings, Detroit, Mich., 1995. Such references, of Estimating a signal to noise ratio for each subband of the 

which the patent is assigned to the assignee of the present signal 

application, are hereby incorporated by reference. 5 ^ method of claim a wherein determinirlg an overall 

In addition, the bias-variance trade-off concept is a related average error for a filtcred spccch signal ^pri^g the 

to the Signal Subspace Technique described in an article by 15 filtered subbands comprises calculating a mean-squared 

Yariv Ephraim and Harry Van Trees entitled "A Signal error. 

Subspace Approach for Speech Enhancement/' IEEE 6. The method of claim 1 further comprising estimating 

ICASSP Proceedings, 1993, vol. II), which is also hereby parameters for the plurality of filters based on a clean speech 

incorporated by reference. The bias-variance trade-off of the signal. 

present invention, however, is a new way of characterizing 20 7. The method of claim 1 wherein the plurality of filters 

this approach. comprises a filter bank. 

The present invention is thus a non-trivial adaptive hybrid 8. The method of claim 1 wherein each of the plurality of 

and extension of RASTA and Signal Subspace techniques filters is associated with one of the plurality of subbands. 

for noise suppression. In contrast to the present invention, 9. A system for adaptively filtering a speech signal, the 

such techniques are, respectively, not adaptive and have 25 svslem comprising: 

always been cast as a reduced rank model rather than a means for decomposing the speech signal into a plurality 

bias-variance trade-off problem. °f subbands; 

As is readily apparent from the foregoing description, means for determining a speech quality indicator for each 

then, the present invention provides an improved method subband; 

and system for filtering speech signals. More specifically, 30 a plurality of filters for filtering the subbands; 

the present invention can be applied to speech signals to means for selecting one of the plurality of filters for each 

adaptively reduce noise in speaker to speaker conversation subband, wherein the filter selected depends on the 

and in speaker to machine recognition applications. Abetter speech quality indicator determined for the subband; 

quality service will result in improved satisfaction among means for determining an overall average error for a 

cellular and Personal Communication System (PCS) cus- 35 filtered speech signal comprising the filtered subbands; 

tomers. means for identifying at least one filtered subband which, 

While the present invention has been described in con- if excluded from the filtered speech signal, would 

junction with wireless communications, those of ordinary reduce the overall average error determined; and 

skill in the art will recognize its utility in any application means for combining, with the exception of the at least 

where noise suppression is desired. In that regard, it is to be 40 one filtered subband identified, the filtered subbands to 

understood that the present invention has been described in provide an estimated filtered speech signal, 

an illustrative manner and the terminology which has been io. The system of claim 9 wherein the means for decom- 

used is intended to be in the nature of words of description posing the signal into subbands comprises means for per- 

rather than of limitation. As previously stated, many modi- forming a transform on the signal to provide approximately 

fications and variations of the present invention are possible 45 orthogonal components. 

in light of the above teachings. Therefore, it is also to be u. The system of claim 10 wherein the means for 

understood that, within the scope of the following claims, performing a transform on the signal comprises means for 

the invention may be practiced otherwise than as specifically performing a discrete Fourier transform on the signal, 

described. 12 The system of claim 9 wherein the means for deter- 

We claim: 50 mining a speech quality indicator for each subband of the 

1. A method for adaptively filtering a speech signal, the signal comprises means for estimating a signal to noise ratio 

method comprising: f or eacn subband of the signal. 

decomposing the speech signal into a plurality of sub- 13. The system of claim 9 wherein the means for deter- 

bands; mining an overall average error for a filtered speech signal 

determining a speech quality indicator for each subband; 55 comprising the filtered subbands comprises means for cal- 

selecting one of a plurality of filters for each subband, bating a mean-squared error 

wherein the filter selected depends on the speech qual- 14 ?» SyStem ° f C l aim u 9 ^'J. comprising means for 

ity indicator determined for the subband; estimating parameters for the plurality of filters based on a 

fl1 . t_ I* j - . . „. . clean speech signal. 

filtering each subband according to the filter selected; 60 15 The &yslem of daim 9 wherein ^ q{ ^ 

determining an overall average error for a filtered speech comprises a filter bank. 

signal comprising the filtered subbands; ig. The system of claim 9 wherein each of the plurality of 

identifying at least one filtered subband which, if filters is associated with one of the plurality of subbands. 

excluded from the filtered speech signal, would reduce 

the overall average error determined; and ***** 
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